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Preface  &  Acknowledgements 


Welcome  to  our  Ninth  Annual  Acquisition  Research  Symposium!  This  event  is  the 
highlight  of  the  year  for  the  Acquisition  Research  Program  (ARP)  here  at  the  Naval 
Postgraduate  School  (NPS)  because  it  showcases  the  findings  of  recently  completed 
research  projects — and  that  research  activity  has  been  prolific!  Since  the  ARP’s  founding  in 
2003,  over  800  original  research  reports  have  been  added  to  the  acquisition  body  of 
knowledge.  We  continue  to  add  to  that  library,  located  online  at 

www.acquisitionresearch.net,  at  a  rate  of  roughly  140  reports  per  year.  This  activity  has 
engaged  researchers  at  over  60  universities  and  other  institutions,  greatly  enhancing  the 
diversity  of  thought  brought  to  bear  on  the  business  activities  of  the  DoD. 

We  generate  this  level  of  activity  in  three  ways.  First,  we  solicit  research  topics  from 
academia  and  other  institutions  through  an  annual  Broad  Agency  Announcement, 
sponsored  by  the  USD(AT&L).  Second,  we  issue  an  annual  internal  call  for  proposals  to 
seek  NPS  faculty  research  supporting  the  interests  of  our  program  sponsors.  Finally,  we 
serve  as  a  “broker”  to  market  specific  research  topics  identified  by  our  sponsors  to  NPS 
graduate  students.  This  three-pronged  approach  provides  for  a  rich  and  broad  diversity  of 
scholarly  rigor  mixed  with  a  good  blend  of  practitioner  experience  in  the  field  of  acquisition. 
We  are  grateful  to  those  of  you  who  have  contributed  to  our  research  program  in  the  past 
and  hope  this  symposium  will  spark  even  more  participation. 

We  encourage  you  to  be  active  participants  at  the  symposium.  Indeed,  active 
participation  has  been  the  hallmark  of  previous  symposia.  We  purposely  limit  attendance  to 
350  people  to  encourage  just  that.  In  addition,  this  forum  is  unique  in  its  effort  to  bring 
scholars  and  practitioners  together  around  acquisition  research  that  is  both  relevant  in 
application  and  rigorous  in  method.  Seldom  will  you  get  the  opportunity  to  interact  with  so 
many  top  DoD  acquisition  officials  and  acquisition  researchers.  We  encourage  dialogue  both 
in  the  formal  panel  sessions  and  in  the  many  opportunities  we  make  available  at  meals, 
breaks,  and  the  day-ending  socials.  Many  of  our  researchers  use  these  occasions  to 
establish  new  teaming  arrangements  for  future  research  work.  In  the  words  of  one  senior 
government  official,  “I  would  not  miss  this  symposium  for  the  world  as  it  is  the  best  forum 
I’ve  found  for  catching  up  on  acquisition  issues  and  learning  from  the  great  presenters.” 

We  expect  affordability  to  be  a  major  focus  at  this  year’s  event.  It  is  a  central  tenet  of 
the  DoD’s  Better  Buying  Power  initiatives,  and  budget  projections  indicate  it  will  continue  to 
be  important  as  the  nation  works  its  way  out  of  the  recession.  This  suggests  that  research 
with  a  focus  on  affordability  will  be  of  great  interest  to  the  DoD  leadership  in  the  year  to 
come.  Whether  you’re  a  practitioner  or  scholar,  we  invite  you  to  participate  in  that  research. 

We  gratefully  acknowledge  the  ongoing  support  and  leadership  of  our  sponsors, 
whose  foresight  and  vision  have  assured  the  continuing  success  of  the  ARP: 

•  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology,  &  Logistics) 

•  Director,  Acquisition  Career  Management,  ASN  (RD&A) 

•  Program  Executive  Officer,  SFIIPS 

•  Commander,  Naval  Sea  Systems  Command 

•  Program  Executive  Officer,  Integrated  Warfare  Systems 

•  Army  Contracting  Command,  U.S.  Army  Materiel  Command 

•  Office  of  the  Assistant  Secretary  of  the  Air  Force  (Acquisition) 
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•  Office  of  the  Assistant  Secretary  of  the  Army  (Acquisition,  Logistics,  & 
Technology) 

•  Deputy  Director,  Acquisition  Career  Management,  U.S.  Army 

•  Office  of  Procurement  and  Assistance  Management  Headquarters,  Department 
of  Energy 

•  Director,  Defense  Security  Cooperation  Agency 

•  Deputy  Assistant  Secretary  of  the  Navy,  Research,  Development,  Test  & 
Evaluation 

•  Program  Executive  Officer,  Tactical  Aircraft 

•  Director,  Office  of  Small  Business  Programs,  Department  of  the  Navy 

•  Director,  Office  of  Acquisition  Resources  and  Analysis  (ARA) 

•  Deputy  Assistant  Secretary  of  the  Navy,  Acquisition  &  Procurement 

•  Director  of  Open  Architecture,  DASN  (RDT&E) 

•  Program  Executive  Officer,  Littoral  Combat  Ships 

We  also  thank  the  Naval  Postgraduate  School  Foundation  and  acknowledge  its 
generous  contributions  in  support  of  this  symposium. 

James  B.  Greene  Jr.  Keith  F.  Snider,  PhD 

Rear  Admiral,  U.S.  Navy  (Ret.)  Associate  Professor 
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Joseph  L.  Yakovac  Jr. — Lt.  Gen.  Yakovac  retired  from  the  United  States  Army  in  2007,  concluding 
30  years  of  military  service.  His  last  assignment  was  as  director  of  the  Army  Acquisition  Corps  and 
military  deputy  to  the  Assistant  Secretary  of  Defense  for  Acquisition,  Logistics,  and  Technology.  In 
those  roles,  Lt.  Gen.  Yakovac  managed  a  dedicated  team  of  military  and  civilian  acquisition  experts  to 
make  sure  America’s  soldiers  received  state-of-the-art  critical  systems  and  support  across  a  full 
spectrum  of  Army  operations.  He  also  provided  critical  military  insight  to  the  Department  of  Defense 
senior  civilian  leadership  on  acquisition  management,  technological  infrastructure  development,  and 
systems  management. 

Previously,  Lt.  Gen.  Yakovac  worked  in  systems  acquisition,  U.S.  Army  Tank-Automotive 
Command  (TACOM),  and  in  systems  management  and  horizontal  technology  integration  for  the 
Office  of  the  Assistant  Secretary  of  the  Army  for  Acquisition,  Logistics,  and  Technology.  He  has  also 
served  as  executive  officer  and  branch  chief  for  the  Bradley  Fighting  Vehicle  and  as  a  brigade 
operations  officer  and  battalion  executive  officer,  U.S.  Army  Europe  and  U.S.  Army  Tank-Automotive 
Command  (TACOM). 

Lt.  Gen.  Yakovac  was  commissioned  in  the  infantry  upon  his  graduation  from  the  U.S.  Military 
Academy  at  West  Point.  He  served  as  a  platoon  leader,  executive  officer,  and  company  commander 
in  mechanized  infantry  units.  He  earned  a  Master  of  Science  in  mechanical  engineering  from  the 
University  of  Colorado  at  Boulder  before  returning  to  West  Point  as  an  assistant  professor. 

Lt.  Gen.  Yakovac  is  a  graduate  of  the  Armor  Officer  Advanced  Course,  the  Army  Command  and 
General  Staff  College,  the  Defense  Systems  Management  College,  and  the  Industrial  College  of  the 
Armed  Forces.  He  has  earned  the  Expert  Infantry  Badge,  the  Ranger  Tab,  the  Parachutist  Badge, 
and  for  his  service  has  received  the  Distinguished  Service  Medal,  the  Legion  of  Merit  three  times  and 
the  Army  Meritorious  Service  Medal  seven  times. 
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Abstract 

Under  support  from  the  Acquisition  Research  Program  and  government/industry 
partnerships,  previous  research  has  successfully  developed,  tested,  and  implemented  a 
system  maturity  measure  (i.e.,  a  system  readiness  level  [SRL]);  supporting  optimization 
models;  and  an  enhanced  SRL  hierarchy  for  multi-function,  multi-capability  (MFMC)  systems. 

The  later  developments  are  predicated  on  what  has  become  the  accustomed  challenge  for 
managers  and  engineers  to  properly  assess  systems’  development  and  acquisition  to  ensure 
the  achievement  of  critical  capabilities  and  functions  while  deciding  among  multiple 
technologies  with  similar  functionalities  but  different  maturity  levels  and  limited  resources. 

Building  on  these  developments,  the  proposed  research  responds  to  the  question:  How  can 
we  efficiently  and  effectively  allocate  available  resources  to  ensure  the  maturity  achievement 
of  critical  functions  and  capabilities  in  a  MFMC  system  when  facing  competing  technology 
alternatives?  As  an  answer,  we  propose  the  development  of  multi-objective  optimization 
models  and  solution  approaches  that  can  be  used  to  evaluate  systems’  development 
maturity,  to  track  progress,  to  identify  component  criticality,  and  to  form  corresponding 
strategies  for  understanding  trade-offs  in  technology  and  integration  options.  Further,  this 
effort  summarizes  and  document  the  previous  and  proposed  research  in  cooperation  with 
industry  and  government  partners  to  produce  a  comprehensive  guidebook  for  public 
distribution  on  the  proper  application  of  this  research  in  methods,  processes,  and  tools  (MPT) 
for  defense  acquisition. 

Executive  Summary 

Through  support  from  the  Naval  Postgraduate  School  (NPS)  and 
government/industry  partnerships,  the  Systems  Development  and  Maturity  Laboratory 
(SysDML)  at  Stevens  Institute  of  Technology  has 

a)  developed  a  methodology  for  determining  a  system’s 
development/acquisition  maturity  (i.e.,  system  readiness  level  [SRL];  Sauser, 
Ramirez-Marquez,  Magnaye,  &  Tan,  2008a,  2008b); 

b)  formulated  two  optimization  models  for  allocating  resources  so  as  to  optimize 
cost,  or  schedule,  or  maturity  performance  (i.e.,  SRLmax  and  SCODmin; 

Magnaye,  Sauser,  &  Ramirez-Marquez,  2010;  Ramirez-Marquez  &  Sauser, 

2009); 

c)  defined  a  methodology  that  combines  items  (a)  and  (b)  into  an  approach 
called  systems  earned  readiness  management  (SERM;  Magnaye,  2010; 

Magnaye,  Sauser,  &  Ramirez-Marquez,  2009); 

d)  developed  models  to  determine  which  components  (understood  as  the 
technologies  that  are  integrations  off  of  which  the  system  is  built)  are 
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sufficient,  critical,  or  important  to  achieve  a  level  of  system  maturity  for  the 
intended  functionalities  and  capabilities  that  can  satisfy  warfighter’s  needs 
(Tan,  Sauser,  &  Ramirez-Marquez,  2011b;  Tan  et  al.,  2010);  and 

e)  proposed  an  enhanced  SRL  hierarchy  that  enables  maturity  assessment  at 
capability,  function,  and  system  levels;  and  incorporated  an  analysis  of 
alternatives  (AoA,  or  trade-off)  for  technology  and  integration  component 
adoption,  which  impacts  the  system  maturity,  functionality,  and  capability 
(Tan  et  al.,  2011a;  Tan,  et  al.,  2011b). 

While  these  efforts  are  necessary  and  relevant  to  the  understanding  of  systems 
maturity  and  alternatives  comparison  and  prioritization,  its  application  is  limited  in  an 
acquisition  situation  where  in  the  procurement  of  a  complex  system  there  are  many 
technology  alternatives  with  the  same  functionality  but  different  maturity  statuses  or  different 
cost  requests  for  further  maturation.  Therefore,  this  research  intends  to  build  on  the  current 
AoA  research  to  propose  the  employment  of  multi-objective  (MO)  optimization  models  for 
systems  of  multiple  functions  and  multiple  capabilities  (MFMC). 


Contrary  to  the  single-objective  optimization  models  developed  in  previous  research, 
this  proposed  effort  will  develop  and  solve  the  MO  problem  and  provide  a  range  of  solutions 
to  assist  managers  with  flexibility  in  the  planning  of  system  acquisition  and  further  system 
maturation.  Thus,  this  research  will  address  a  fundamental  question:  How  can  we  efficiently 
and  effectively  allocate  available  resources  to  ensure  the  maturity  achievement  of  critical 
functions  and  capabilities  in  a  MFMC  system  when  facing  competing  technology 
alternatives?  As  a  summary  of  the  efforts  of  the  SysDML,  Figure  1  summarizes  previous 
research,  how  it  relates  to  the  proposed  research,  and  a  guidebook  under  development  by 
the  SysDML  that  explains  the  theory  and  application  of  this  research. 


Figure  1.  Evolutionary  Development  Plan 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-49- 


References 

Magnaye,  R.  (2010).  Earned  readiness  management  for  scheduling,  monitoring,  and  evaluating  the 
development  of  systems.  Hoboken,  NJ:  Stevens  Institute  of  Technology. 

Magnaye,  R.,  Sauser,  B.,  &  Ramirez-Marquez,  J.  E.  (2009).  Using  a  system  maturity  scale  to  monitor 
and  evaluate  the  development  of  systems.  Paper  presented  at  the  Sixth  Annual  Acquisition 
Research  Symposium,  Monterey,  CA. 

Magnaye,  R.,  Sauser,  B.,  &  Ramirez-Marquez,  J.  (2010).  System  development  planning  using 

readiness  levels  in  a  cost  of  development  minimization  model.  Systems  Engineering,  13(4),  31 1- 
323. 

Ramirez-Marquez,  J.  E.,  &  Sauser,  B.  J.  (2009).  System  development  planning  via  system  maturity 
optimization.  IEEE  Transactions  on  Engineering  Management,  56(3),  533-548. 

Sauser,  B.,  Ramirez-Marquez,  J.,  Magnaye,  R.,  &  Tan,  W.  (2008a).  System  maturity  indices  for 

decision  support  in  the  defense  acquisition  process.  Paper  presented  at  the  Defense  Acquisition 
Research  Symposium,  Monterey,  CA. 

Sauser,  B.,  Ramirez-Marquez,  J.,  Magnaye,  R.,  &  Tan,  W.  (2008b).  A  systems  approach  to 

expanding  the  technology  readiness  level  within  defense  acquisition.  International  Journal  of 
Defense  Acquisition  Management,  1,  39-58. 

Tan,  W.,  Sauser,  B.,  &  Ramirez-Marquez,  J.  (2011a).  Analysis  of  alternatives  in  system  capability 
satisficing  for  effective  acquisition.  Paper  presented  at  the  Eighth  Annual  Acquisition  Research 
Symposium,  Monterey,  CA. 

Tan,  W.,  Sauser,  B.,  &  Ramirez-Marquez,  J.  (2011b).  Analyzing  component  importance  in 

multifunction  multicapability  systems  developmental  maturity  assessment.  IEEE  Transaction  on 
Engineering  Management,  58(2),  275-294.  doi:  10.1 109/TEM. 2010. 2071877 

Tan,  W.,  Sauser,  B.,  Ramirez-Marquez,  J.,  Magnaye,  R.,  Nowicki,  D.,  &  Deshmukh,  A.  (2010). 

System  capability  satisficing  in  defense  acquisition  via  element  importance  measures.  Paper 
presented  at  the  Seventh  Annual  Acquisition  Research  Symposium,  Monterey,  CA. 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-50- 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 

555  DYER  ROAD,  INGERSOLL  HALL 

MONTEREY  CA  93943 

www.acquisitionresearch.net 


Analyzing  Component  Importance  in 
Multi-Function  Multi-Capability  Systems 
Developmental  Maturity  Assessment 


Dr.  Brian  Sauser 

Associate  Professor 

Stevens  Institute  of  Technology 

School  of  Systems  &  Enterprises 

bsauser@stevens.edu 


Stevens 


INSTITUTE  of  TECHNOLOGY 


School  of 

Systems&Enterprises 


©2011  Stevens  Institute  of  Technology 


Visit  us  at  http://www.SysDML.com 


Emotional-Functional 


Indicates  how  a  system  responds  to  the 
circumstances  or  environment  in  an 
appropriate  and  adaptive  manner. 


Physical-Physical 


This  response  is  designed  (in  some 
instances  learned)  and  not  determined 
by  the  system’s  age. 


Encompases  being  aware  of  the  correct 
time  and  place  to  deploy  and  knowing 
when  to  operate  appropriately 
according  to  the  situation 
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Systems  Evolution  and  Lifecycle  Management 


Value  Proposition: 

■To  provide  a  systemic  view  of 
development  maturity  with 
opportunities  to  drill  down  to 
element-level  contributions 
■To  allows  managers  to  evaluate 
system  development  to  take 
proactive  measures 
■To  create  highly  adaptive 
methods,  processes,  and  tools  to 
use  on  a  wide  array  of  system 
engineering  development  efforts 
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How  to  Use  this  Template 


1-input  the  date;  2-move  the  selector  in  the 
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check  the  CAPABILITY  of  interest;  5-for  $ 
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►  This  is  RESEARCH... 


•  Component  importance  is  significantly  dependent  on  interest : 
component  importance  should  be  situation-dependent 

•  Discernibility  in  components  importance  determination :  difference 
between  important  components  is  very  small,  largely  due  to  the 
inconsistency  of  estimates  across  all  technologies 

•  Technologies  that  have  more  integration  rank  relatively  higher 
than  those  having  less  integration  when  all  other  factors  are  the 
same:  With  out  a  proper  weighting  of  component  importance, 
intergration  will  drive  risk 

•  Maturity  of  one  system  is  not  the  same  as  maturity  for  another 
system:  SRL  for  one  system  cannot  be  compared  to  the  SRL  of 
another  system  unless  they  are  the  same  system. 
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